Skip to content

Shared state + cache reuse: Postgres backend, adapter-aware caching, state inspection - #10

Merged
iamgp merged 1 commit into
mainfrom
feat/shared-state-cache
Sep 15, 2026
Merged

iamgp merged 1 commit into
mainfrom
feat/shared-state-cache

Conversation

@iamgp

@iamgp iamgp commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Bundle 6 — shared state + cache reuse.

  • PostgresStateStore — full StateStore implementation mirroring the SQLite schema (upserts, BIGSERIAL ordering tiebreak, same migrations). The postgres client drives an internal Tokio runtime that cannot block_on inside the outer one, so the client lives on a dedicated worker thread and each trait call ships a closure over a channel — legal from sync and async callers alike.
  • --state / PHLO_STATE_URL — postgres:///postgresql:// selects the shared backend; any other value is a SQLite file path (default .phlo/transform/state.db). An explicitly configured location that fails is a hard error — never a silent fallback to local state — and credentials in the URL are stripped before it appears in errors or doctor output.
  • Safe cross-environment cache reuse — a same-version record in another environment authorises Cached only when it names the same physical target relation AND was produced by the same adapter; a same-version record failing either check yields build with a new cache_miss reason (detail names the mismatch), and a current-environment record produced by a different adapter rebuilds with adapter_change. model_versions records the producing adapter (best-effort ALTER TABLE migration; NULL rows cannot authorise reuse).
  • phlo-transform state — read-only inspection: runs (env filter), show <run-id|prefix> (model/seed/test records), model <name> (version components, target, adapter), promotions. All honour --json. RunSummary gains environment.
  • Concurrency — SQLite busy_timeout(5s) so parallel processes sharing a state file wait out lock contention.

Test plan

  • cargo fmt --all -- --check clean
  • cargo clippy --workspace --all-targets -- -D warnings clean
  • cargo test --workspace green — incl. new: cache_reuse_requires_same_physical_target, cache_reuse_requires_same_adapter, cache_reuse_rejects_unrecorded_adapter, materialisation_by_another_adapter_rebuilds, concurrent_sqlite_writers_complete, state_commands_inspect_recorded_runs_on_duckdb, state_flag_selects_sqlite_path, unreachable_postgres_state_url_errors
  • Postgres backend covered by postgres_roundtrip_run_and_versions (opt-in via PHLO_TEST_POSTGRES_URL; skips cleanly without a server)

@iamgp
iamgp force-pushed the feat/shared-state-cache branch from 96766f5 to 7797a52 Compare September 14, 2026 08:29
@iamgp
iamgp changed the base branch from main to feat/promotion-provenance September 14, 2026 08:29
@iamgp
iamgp force-pushed the feat/shared-state-cache branch 2 times, most recently from 6c5938b to d2c7733 Compare September 14, 2026 09:53
Base automatically changed from feat/promotion-provenance to main September 14, 2026 10:32
@iamgp
iamgp force-pushed the feat/shared-state-cache branch 2 times, most recently from 86d2e38 to 2668c66 Compare September 14, 2026 12:19
…re caching, state inspection

- PostgresStateStore: full StateStore impl behind a dedicated worker
  thread (the postgres client owns an internal Tokio runtime that cannot
  block inside the outer one); schema mirrors SQLite with upserts and an
  identity column for deterministic newest-first ordering.
- --state / PHLO_STATE_URL: postgres:// selects the shared backend, any
  other value is a SQLite file path; explicit locations fail hard, never
  silently fall back, and credentials are redacted from errors/doctor.
- Safe cross-environment cache reuse: a same-version record authorises
  Cached only when it names the same physical target AND was produced by
  the same adapter; otherwise Build with cache_miss. Current-env records
  from another adapter rebuild with adapter_change. Records persist the
  producing adapter (model_versions.adapter, best-effort migration).
- phlo-transform state: runs / show <id|prefix> / model / promotions,
  human + --json. RunSummary gains environment.
- Concurrency: SQLite busy_timeout(5s) so parallel processes on one
  state file wait out lock contention.
@iamgp
iamgp force-pushed the feat/shared-state-cache branch from 2668c66 to a22e5ba Compare September 14, 2026 13:25
@iamgp
iamgp added this pull request to stack #15 September 15, 2026 06:48
@iamgp
iamgp merged commit 709e3d0 into main Sep 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant